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Abstract 

In this paper we study the relationship between the optimal value of a homogeneous quadratic 
optimization problem and that of its Semidefinite Programming (SDP) relaxation. We consider 
two quadratic optimization models: (1) min{a;*Ca; | x* A^x > 1, x € F™, k = 0, 1, ...,m}; and (2) 
max{x*Ca; | x* A^x < 1, x G F™, k = 0, 1, ...,m}. If one of A^s is indefinite while others and C 
are positive semidefinite, we prove that the ratio between the optimal value of (1) and its SDP 
relaxation is upper bounded by 0(m 2 ) when F is the real line R, and by 0(m) when F is the 
complex plane C. This result is an extension of the recent work of Luo et al. [8]. For (2), we 
show that the same ratio is bounded from below by 0(1/ log m) for both the real and complex 
case, whenever all but one of A^s are positive semidefinite while C can be indefinite. This result 
improves the so-called approximate S-Lemma of Ben-Tal et al. [2]. We also consider (2) with 
multiple indefinite quadratic constraints and derive a general bound in terms of the problem data 
and the SDP solution. Throughout the paper, we present examples showing that all of our results 
are essentially tight. 
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1 Introduction 



We consider in this paper homogeneous quadratic optimization problems in either the minimization 
form 

min x*Cx 

s.t. x*A k x > 1, k = 0, 1, ...,m (1.1) 

x e F n , 

or the maximization form 

max x*Cx 

s.t. x*A k x < 1, k = 0, l,...,m (1.2) 

x E F n , 

where matrices Afc and C are n x n, F can be the field of real numbers 1R or the field of complex 
numbers C, and the superscript * represents Hermitian transpose (or regular transpose in case of real 
numbers). Both of above quadratic optimization problems are NP-hard 0[2], even when all the data 
matrices, C and A k , k = l,...,m, are positive semidefinite. Homogeneous quadratic optimization 
problems (|l.ip -( fL2~j) arise naturally in telecommunications and robust control applications; see (8J [2] 
and the references therein. A popular approach to approximately solving the NP-hard quadratic 
programs (|l.lj )- (|1.2j) is to use the so-called Semidefinite Programming (SDP) relaxations as follows: 

min Tr(CA) 

s.t. Tr(A k X) > 1, k = 0,l,...,m 

X G SF™, 

and, respectively, 

max Tr(CX) 
s.t. Tr (A k X) < 1, A; = 0, 1, ...,m 

x e §F™ , 

where Tr(-) represents the trace of a matrix, SF™ denotes the convex cone of positive semidefinite 
matrices in the space of all (Hermitian) symmetric matrices SF n . The above two SDPs are convex and 
can be solved efficiently via interior point methods. After the SDP relaxation problems are solved, we 
can apply a randomization procedure to the corresponding optimal SDP solutions to extract rank-one 
feasible solutions for (jl.ip and (jl.2p respectively. Theoretically, even though the randomized solutions 
obtained in this manner are not globally optimal for either (jl.lj) or (11. 2j) . they can be shown to be 
high quality approximate solutions; see, e.g. Specifically, Nemirovski et al. [9] proved that 

for the maximization problem (jl.2h . if all A k s are positive semidefinite, then the ratio between the 
optimal value of the SDP relaxation problem and that of the original quadratic problem is bounded 
above by O(logm). More generally, Ben-Tal et al. [2] established a so-called approximate S-Lemma 
which shows that the approximation ratio for the SDP relaxation is at most 0(log(n 2 m)) when all 
but one of the matrices A k , k = 0, 1, m are positive semidefinite. 

In a parallel development, Luo et al. [8 J considered the homogeneous quadratic optimization in 
minimization form (jl.ip . It turns out that the SDP approximation ratio for the minimization version 
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of the problem takes a quite different form as compared with its maximization counterpart. When 
all the matrices and C are positive semidefinite, Luo et al. [8] showed that the ratio between 
the original optimal value and the SDP relaxation optimal value is bounded above by 0(m 2 ) when 
F = R and by 0(m) when F = C. All these bounds are shown to be tight in the worst case, although 
the simulation studies in [8] showed that the ratios are typically close to 1. In other words, the 
average performance can be much better than the stated worst-case bounds for randomly generated 
instances. Recently, So et al. [llj developed methods for finding approximate low rank solutions for 
linear matrix inequalities. Their results unify the approximation bounds of Nemirovski et al. [9] and 
Luo et al. [8] as special cases (rank being 1), when all the data matrices are positive semidefinite. 

In this paper, we study the approximation ratio of the SDP relaxation for homogeneous quadratic 
optimization problems (|1 .ip — ([1.2)1 when some of the constraint matrices {A^} are indefinite. Our 
results are as follows. In Section [3l we show that, for the problem in minimization form (jl.lj) . 
the upper bounds for the approximation ratios of the SDP relaxation as presented in [8] (0(m 2 ) 
and 0(m) for F = R and F = C respectively) hold true even when one of the constraint matrices is 
indefinite. If there are more than one indefinite quadratic constraints, we show by an example that the 
approximation ratio can be infinite. Therefore, our bounds are essentially best possible. In Section SI 
we consider the problem in maximization form ([1.2ft . We improve the approximate S-Lemma of Ben- 
Tal et al. [2] by reducing their upper bound on approximation ratio from 0(log(n 2 m)) to O(logm) 
when one quadratic inequality is indefinite. In the process of establishing this new bound, we resolve 
a conjecture by Ben-Tal et al. [2] on a possible universal lower bound for the probability that a 
homogeneous quadratic form of binary i.i.d. Bernoulli random variables lies below its mean. Finally, 
in Section [5] we present a new and unifying upper bound on the ratio of the optimal value of SDP 
relaxation over that of the original quadratic maximization problem (jl.2ft without any definiteness 
assumptions. This new general bound involves the problem data and the SDP optimal solution, 
which are computable in polynomial time. We also present an example showing that this bound is 
essentially tight. 



2 Estimating Asymmetry of a Random Variable About its Mean 

To facilitate the technical analysis in subsequent sections, we establish in this section a bound on the 
probability for a general random variable to be above (or symmetrically, below) its mean value, using 
only the high order moment information of the random variable. This problem is of importance on 
its own in statistics and probability theory. The following lemma is a generalization of Theorem 2.1 
in0. 

Lemma 2.1. Suppose that a random variable $ satisfies E<3? = 0, Var(<I>) = 1 and E|$|* < r for 
some t > 2 and r > 0. Then Prob{$ > 0} > 0.25t~"^ an d Prob{$ < 0} > 0.25r~^2. 

Proof. Let p\ = Prob > 0} and p 2 = Prob {<£ < 0}. Also let Y\ = max($, 0) and Y 2 = — min($, 0). 
Since E<I> = 0, we know EYi — EY 2 = 0. Let s := EYi = EY 2 . By Holder's inequality it follows that 
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( E jrt)l/(t-l)( Eyi )(t-2)/(t-l) > E y2 and ( E y^i/(t-i)( E y 2 )(*-2)/(t-i) > E y2 Since E ^t + E1 ^ = E |$|* ; 
we have 

r > E|*|* = EY? + EY 2 * > ^YiY' 1 + ( EY 2) t ~\ 

Let u = EY 2 G [0,1]. Since EY 2 + EY 2 2 = E$ 2 = Var($) = 1, it follows that s*" 2 > " t " 1 +(i-") t ~ 1 , 
On the other hand, by the Cauchy-Schwartz inequality, we have 

s 2 = (EYi) 2 = (E(sign(Y 1 )Y 1 )) 2 < E(sign(Y 1 ) 2 )EY 2 < Pl u 

which implies that 

V -1 + (1 - it)*- 1 ' 



pi > u 1 



+ (1 - u)*" 1 )'- 2 __L_ 



> (n*- 1 + (i -uy- 1 )*- 2 T -~—> 



= 0.25r"^, 

where the third inequality follows from the convexity of the function u* -1 when t > 2. Obviously, 

2_ 

the equality can not hold throughout. Therefore, p\ > 0.25r *- 2 . By symmetry, we also have 
£> 2 >0.25t~^. □ 

In case t = 4, Lemma I2TT1 asserts that Prob{<!> > 0} > j- and Prob{<!> < 0} > j-. However, in this 
particular case, this specific bound can in fact be further sharpened. 

Lemma 2.2. Suppose that a random variable satisfies E<I> = 0, Var(<I>) = 1 and E<I> 4 < r. Then 
Prob{$ > 0} > 2^=3 > gg- and Prob{$ < 0} > > ^7. 

Proof. It follows from the proof in the Lemma 12.11 that 

u 3 + (l-u) 3 (\ \ 1 2^/3-3 9 
Pi > ~ = - + 3n - 3 - > > — . 

TU \U J T T ZUT 

By symmetry, P2 > ^ holds as well. □ 



3 Homogenous Quadratic Minimization and SDP Relaxation 

Consider the homogeneous quadratic optimization 

w™ m := min x*Cx 

s.t. x*Aj t x > 1, k = 0, 1, m (3-1) 

x G F n , 
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where C, Ai,A%, A m £ SF™' are symmetric matrices. This problem is generally NP-hard [8]. A 
natural semidefinite programming (SDP) relaxation to the above quadratic optimization problem is 

<J := min Tr (CZ) 

s.t. Tr (A k Z) > 1, k = 0,1, ...,m (3.2) 

z e sf" . 

Obviously, the SDP relaxation provides a lower bound, i.e., v™J^ < v™ n . In the case C = I n , and 
A ,Ai, A m are all positive semidefinite, Luo et al. [8] proved that v q I p m /v™p < 27 ("^ +1 ) f or F = R, 
and v^p m /v^p < 8(m + 1) for F = C. Moreover, when there are two or more of Aq, A±, A m 
are indefinite, there is in general no data-independent upper bound on v q r p in /v^p, as shown by the 
following example [8]: 

2 i 2 

mm x 1 + x 2 
s.t. x\ > 1 

x 2 + Mxix 2 > 1 

X 2 — MX1X2 > 1 

where M > is a constant. In the above example, = 1, and the last two constraints imply 
x\ > M|xi||x2| + 1 which, together with the first constraint x\ > 1, yield x\ > M|xi| + 1 or, 
equivalents, \x x \ > (M+y/M 2 + 4)/2. Therefore, f™ n > 1 + |(M+VM 2 + 4) 2 . That is, v^/v^ > 
1 + i(M + VM 2 + 4) 2 , which can be arbitrarily large, depending on the problem data M > 0. 

In this section, we consider the homogeneous quadratic optimization (|3.ip under the assumption that 
C, Ai, A 2 , A m G SF" are positive semidefinite while Aq £ SF™ can be indefinite. Throughout 
this section, we assume that (13. 1 h is feasible, and that there is /x^ > 0, /c = 0,1,..., to, such that 
SfcLo f^kAk ~< 0. This assumption guarantees that the SDP relaxation is primal feasible while its 
dual problem satisfies the Slater condition. Hence the strong duality holds and the primal problem 
(|3.2|) has an optimal solution that attains its infimum. 

Our analysis shall treat the cases F = R and F = C separately, leading to different bounds and 
flavors. For clarity, the analysis will be presented in the next two subsections. 

3.1 The real case 

Let us start with a useful lemma regarding a lower bound on worst asymmetric mass distributions 
for a ^-distribution around its mean vector. In fact this result is interesting on its own right. 

Lemma 3.1. Let T{ be any real numbers, i = l,...,n, and let r\ ~ N(0,I n ) be an n- dimensional 
normal distribution with zero mean and covariance matrix I n . Then we have 

Prob jl>(^ " X ) ^ °} > i5o' Pr ° b {Pj 1 ^ " 1} " °} > m 
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Proof. Note that E(rg - l) 2 = E(r]f - 2rjf + 1) = 3 - 2 + 1 = 2. Let * = EiLi 7 "^ - 1), and 
$ = —j=^= — t . Then E$ = and Var($) = 1. Since E(rjf - l) 2 = 2, and E(r? 2 - l) 4 = 60, direct 



L«i=i r i 

calculation shows 

n / n \ 2 / n 

E* 4 = 48^ + 12 hTr 2 <6oK> 2 

i=l \i=l / \i=l 

Therefore, we have 



Eg* = E ^ < 15 

4(Er=i^) 2 - 

It follows from Lemma [231 that Prob{$ > 0} > Similarly, we have Prob{<!> < 0} > ^ by 

symmetry. □ 

Using Holder's inequality, we also have E|^| 3 < 60j(^™ =1 ff)* and E|<3?| 3 < 15* which can be used to 
lower Prob{<£ > 0} (c.f. Theorem 2.1 in [7]). However, in this particular case, the bound so obtained 
is slightly worse than the one that we derived in Lemma 13. 11 

Lemma 3.2. Let A, Z be two real symmetric matrices with Z >z and Tr (AZ) > 0. Let £ £ iV(0, Z) 
be a normal random vector with zero mean and covariance matrix Z . Then for any < 7 < 1 we 
have 

Prob{e T ^ < -yE(?M)} < 1 " 

Proof. Let r = rank(^4Z), and Q E R nxn be an orthogonal matrix such that 

Q T (Z 1 2AZ 1 2)Q = diag(Ai,--- ,A r ,0,--- ,0). 

Since Tr (AZ) > we have Ya=i ^* — 0- Let £ £ N(Q, ^n) an d £ := Z?Q£. Then £ follows a Gaussian 
distribution N(0,Z). Moreover, we have £ T A£ = ^[ =1 Aj£ 2 , where £j, i = l,...,r, are independent 
and follow the normal distribution N(0, 1). Therefore, we have E(£ T A£) = El=i ^« an d 

( r r 1 

Prob {fAi < 7 E(e T A£)} = Prob lJ2 X ^ < 7 E A * f 

U=l i=l J 

= Prob^A i (e7-l)<(7-l)E A 



i=l i=l 
r 



< Prob ^Ai(^-l) <0 < 1 



. i=i 



3 

Too' 



where the first inequality follows from 7 S [0, 1] and Yll=i — 0, and the last step is due to 

Lemma 13.11 □ 

Now we are ready to establish the following quality bound for the SDP relaxation. The argument 
follows closely those of |S]. 
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Theorem 3.3. Consider the real quadratic program <\3.1\i and its SDP relaxation where F 

Then, there holds 



v™ n 10 6 m 2 



Proof. Let Z be an optimal solution of the SDP relaxation (|3.2p with rank r satisfying ^ r ^ 2 1 ^ r < m. 
The existence of such matrix solution is well known; cf. Pataki [TO]. Moreover, this low rank matrix 
can be constructed in polynomial-time; cf. [6J. Clearly, r < \j2m. Since Z is feasible, Tr (AqZ) > 1. 
For any < 7 < 1 and /i>0we have 

Prob { min £ T A k £ > 7, £ T C£ < fj, Tr (CZ) 

[0<k<m 

= Prob [fA k £ > 7 for all k = 0,1,..., m, and £ T C£ < /iTr(CZ)} 

> Prob {C T ^C > 7Tr(^ fc i') for all k = 0,1, ...,m, and £ T C£ < ^Tr(CZ)} 
= Prob {£ T A k £ >"/E{£A k £) for all k = 0,1,..., m, and £ T C£ < ,uE(£ T C£)} 

m 

> Prob {fA k i < 7 E(^ fc 0} " Prob {£ T C£ > p E(£ T C£)} . 

fe=0 

Since A k >z for k = 1, ...,m, it follows from Lemma 3.1 of [8] that 

Prob {e T ^ fc e < 7E(e T A fc 0} < max j^, ^"l^ } " 
Although j4o is indefinite, we can use Lemma 13.21 to obtain 

Prob {fAot < jE(fA 0} < 1 - yj- 
Also, since C ^ 0, we can apply Markov inequality to obtain 

Prob {i T Ci > [i E(£ T C0 } < -• 

H 

Combining the above estimates yields 

Prob ( min f A k i > 7, £ T C£ < (iTr (CZ)\ > — - mmax j^/7, 2{j ~ ^ 7 I ' 



0<fc<m J 100 { 7T-2 ) \X 

Let = 100 and 7 = 10 i m -2 ■ Since r < y / 2m, we have > 2 ^_^ 7 ■ For these values of ,u and 7, we 
have 

3 f _ 2(r-l) 7 \ 1 3 ^ 1 1 

m max < -.Ay, > = m > . 

100 \ v " vr-2 J /x 100 100m 100 500 

Therefore, there exists a vector £ G M n such that 

£ T ^fc£>7, fe = 0,1, ...,m, and £ T C£ < (iTr (CZ). 
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Now let x = Then, x T Aj t x > 1, k = 0, 1, to, and 

which establishes the desired bound □ 



3.2 The complex case 

Recall that the density function of a complex- valued normal distributiorj^l r] ~ N c (0, 1) is 

7T 

In polar coordinates, the density function becomes 

-e~ p \ Vpe [0,+oo), [0,2vr). 

7T 



The argument is uniformly distributed in [0, 2tt), and the modulus p has the distribution 

M 



2pe-P , if p > 0; 
0, ifp<0. 



Thus squared modulus \i]\ 2 has the exponential distribution 

Prob{|r?| 2 < a} < 1 - e" a . 

Lemma 3.4. For any real numbers r^, and i.i.d. exponential random variables rji with unit variance, 
i = l, ...,n, there holds 

Prob lj2ji(Vi ~ 1) > o| > ^, Prob |X>^ " 1} " °} > 4' 

Proo/. Note that E(^ - l) 2 = 1. Let $ = ^" =1 - 1) and $ = -/=f — f . Clearly, E$ = and 
Var(<l>) = 1. Since E(r/j — l) 4 = 9, direct calculation shows 



i=l \i=l / \i=l / 

This further implies 

E** = — !^< 9 

Using Lemma 12.21 we have Prob{$ > 0} > i. Similarly, Prob{$ < 0} > i. □ 



1 For a discussion on the complex normal distribution and the related references, see Zhang and Huang [13] 



s 



Interestingly, it is possible to find a closed formula (see e.g. [I] and [T]) for the above probability. In 
particular, if all the Tj's are distinctive, then 



_ j_ 
e T i 



.i=i ) i=i 



Therefore, we have 

e T i 19 
< 



90 < r , , 



20 (i_h.\ 20 

for any distinctive real values Tj, i = 1, ...,n. 
In fact, we conjecture that the following tighter inequalities 



i 



hold for any real values r», i = l,...,n. Inequality (|3.3p can be shown to hold for n = 2,3. It also 
admits a geometric interpretation. Specifically, let us consider the joint exponential distribution on 
]R™ with density e~'^ i =^ Xi . Then, the mean vector of this distribution, or equivalently, the center of 
gravity of M™ is x c := (1, 1, 1) T . Given any real numbers Ti, i = 1, n, the set 



£>(r,;-l) = 



H = { (r/i,?72,...,??„) T i 

' i=l 

represents a hyperplane passing through x c . If we let denote the half space in M. n created by the 
positive side of TL, then inequality (|3.3|) can be interpreted as follows: 



Prob(R™ n H.+) > e , for any hyperplane TL passing through 



x . 



Interestingly, the well-known theorem of Griinbaum [S] can also be viewed from this perspective: for 
any bounded convex body C C M n , if we assign the uniform distribution to C, then the mean vector 
of this distribution is given by the center of gravity 

x c = / dx' 

Volume(C) J c ' 

as a result, if we consider any hyperplane H passing through x c and let Ti. + denote the positive side 
of the hyperplane, then Griinbaum inequality 

Volume (C n H+) > e' 1 Volume (C) 

can be written as 

Prob(C n W+) > e _1 , for any hyperplane TL passing through x c . 

Thus, inequality (|3.3[) can be viewed as an extension of Griinbaum's theorem to the exponential 
distribution over the unbounded convex set C = Wf. 
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Lemma 3.5. Let A, Z be two Hermitian matrices satisfying Z y and Tr (AZ) > 0. Let £ ~ N c (0, Z) 
be a complex normal random vector. Then, for any < 7 < 1, we have 

Prob{£M£< 7 E(£*A£)}<l-i 

Proof. Let Q G C nxn b e an unitary matrix such that 

Q*(Z^AZ^)Q = dmg(X 1 ,--- , X r , 0, • • • , 0) 

where r = rank(AZ). Since Ti (AZ) > 0, it follows that Aj — 0- Let £ G C n be a random 

Gaussian vector drawn from the complex normal distribution N c (0,I n ). Then the random vector 
£ = Z2Q!; follows the Gaussian distribution N c (0, Z). As a result, there holds 



Prob {CM < 7 E(eMf)) = Prob ^ \|^| 2 < 7 ^ A, I 

li=l i=l J 

( n n 

= Prob^A,(|| J | 2 -l)<( 7 -l)^A i 

I i=l i=l 

< Prob i^Aidlil 2 -!) <0 



. i=l 



where the last step follows from 7 G [0, 1] and Yll=i > 0- Since |£j| 2 is exponentially distributed, 
by Lemma 13.41 we have 



Prob j^al^-l^oj^ 



which proves the lemma. □ 
Theorem 3.6. Consider (TOj) and (TO) , w/zere F = C. T/ien 



< 2400m. 



Proof. It is known that in this case, if is finite and m < 3, then v™™/v™J^ = 1 (cf. e.g. [6] 
and [E])- Below we shall only consider the case where m > 4. Let Z be a low rank optimal solution 
of the SDP relaxation (|3.2|) . such that r = rank(Z) < (see [6], §5). The feasibility of Z implies 
that Tr (AqZ) > 1. Similar to Theorem 13.31 we can use the union bound to obtain the following 
inequality 

Prob { min f A fe £ > 7, C'C£ < ^Tr (CZ) 

{0<k<m 

m 

> 1 - Prob {e*A fc C < 7 E(£*4fc0} " Pro b > fi E(CCQ} • 

fc=0 
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Since y 0, k = 1, ...,m, it follows from Lemma 3.4 in [8] that 

Prob {?A h £ < 7 E(r^)} < max || 7j 16(r - 1) 2 7 2 

Although j4o is indefinite, Lemma 13.51 asserts that 

Prob {£M £< 7 E(tA)0}<l-^- 
Therefore, combining these estimates and using Markov inequality, we have 



1 f4 .., 2 1 1 



Prob < min 

r^fcC > 7, £*Cf < /i, TV (CZ) } > m max <^ - 7 , 16(r - 1) V , 

[0<fc<m J 20 [3 J /i 

Now choose /i = 60 and 7 = In this case, | 7 > 16(r — 1) 2 7 2 . We also have a strict lower bound 
of the above probability 

Prob { min £*A fc £ > 7 , £*C£ < ^TV(CZ)} > 0. 

|^0<fc<m J 

This implies that there exists £ 6 C n such that 

£M & £> 7 , & = 0,l,...,m; e*C£< M Tr(CZ). 
Now let x := -^£- Then x* A^x > 1, k = 0, 1, ...,m, and so 

< n < **Cx < ^ < ^jp) = 2400m • Us % n . 
The theorem is proven. □ 

Notice that there are examples (see [8]) which show that the worst-case ratios of v^ n /v™p are indeed 
0(m 2 ) and 0(m) in the real and complex case respectively, even in the absence of indefinite constraint 
x*Aqx > 1. Thus, the bounds of Theorems 13.31 and 13.61 are essentially tight. 

Finally, we may also wonder what happens if there are more than one indefinite quadratic constraint. 
The following example shows that in this case the SDP relaxation does not admit any finite quality 
bound. 



Example 3.7. 



min x\ 

S.t. X\X2 + x\ + x\ > 1 
-X\X2 + x\ + x\ > 1 

2 X 1 ~~ X 3 — 1 
2 X 2 ~ x 3 — 1 
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The first two constraints are equivalent to |xiX2| < x\ + x\ — 1. At the same time, the last two 
constraints imply \x\X2\ > 2(x| + 1). Combining these two inequalities yields 

xl + xl-l>2(xl + l), 

which further implies x\ > 3. Therefore, we must have t>™ in > 3 in this case. However, 

" 4 " 
4 
10 


is feasible for the corresponding SDP relaxation problem and attains an objective value of 0. Thus, 
it must be optimal and thus vfj^ = 0. Hence, v^/vfj^ = oo in this case. 



4 Quadratic Maximization and the Approximate 5-Lemma 

In this section, we consider the nonconvex homogeneous quadratic optimization in the maximization 
form 

v qp X := max x*Cx 

s.t. x*A k x < 1, k = 0, 1, ...,m (4.1) 

x G F n , 

where A k G SF+, k = l,...,m, are positive semidefinite, while C,Aq G §F n may be indefinite. For 
convenience, from now on we shall focus on the case F = W 1 . Unlike the case of minimization form, 
this choice does not significantly affect the quality of SDP approximation ratios, since in the complex 
case the bounds are of the same order of magnitude. We assume that there is fi k > 0, k = 0, 1, m, 
such that 

m 

^2fj, k A k y 0. 

Under this condition, the SDP relaxation satisfies the dual Slater condition. Thus the primal-dual 
optimal solutions exist and the primal-dual optimal objective values are attainable. Let the SDP 
relaxation optimal value be 

^max . = max Tr (c-jq 

s.t. Tr(A k X) < 1, k = 0,1, ...,m (4.2) 

x y o. 

Obviously t>™ ax < v™*. 

Lemma 4.1. Let Wij (1 < i < j < n) be any real numbers, and £j (1 < i < n) be random variables 
such that Prob = — 1} = Prob {£j = 1} = 0.5. Then there holds 



Prob < 



l<i<j<n 
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Proof. Let * = Y,i<i<j<n w iMr Then E * = °> E(* 2 ) = £i<i<j<n<4 and 

E(* 4 ) = £ 4 + 6 £ u£«& + W 

l<i<j<n (i,j)<(k,£) 

where < (k, I) means i < k or i = k and j < I, and 

W = 24 (wijWi k w j£ w k e + WijWiiw jk w k i + w ik w if w jk w^) 

l<i<j<k<£<n 




Therefore we have E(\& 4 ) < 39(J2i<i< <n wf) 2 , since 



l<i<j<n w ijJ > 

- 

l<i<i<n (*>j)<(M) \l<i<i<" 



*4 



Now let $ = , S =. Then E($) = 0, Var($) = 1 and E($ 4 ) < 39. By LemmaES we have 

V/Z]l<i<j<n W ij 



Prob {$ < 0} > — . 
L ~ J 87 

The desired result follows. □ 

Lemma 14.11 settles in the affirmative an open question of Ben-Tal et al. Conjecture A. 5] who 
conjectured that 



\l<i<j<n I 



except that we have a smaller constant of 1/87. The above inequality was needed to establish the so 
called approximate S-Lemma — an extension of the well-known S'-Lemma, which is important in the 
context of robust optimization and is closely related to our analysis in this section. In their work [9j, 
Ben-Tal et al. derived a weaker lower bound of l/8n 2 , which goes to zero as n — > oo. 

We can now use Lemma [4. II to analyze the performance of SDP relaxation for (|4,2p . Let X = UU T be 
one optimal solution of (|4,2p . where U G R nxr and r = rank(X). Suppose Q £ M nxr is the orthogonal 
matrix such that C := Q T U T CUQ is diagonal. Let k = 1, ...,r, be i.i.d. random variables taking 
values —1 or 1 with equal probabilities, and let 

x(0 := ; 1 UQ£, 
max £ T A k £ 

0<k<m 
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where A k = Q T U T A^UQ. Note that the above random vector x(£) is always well-defined, since the 
assumption J2T=o ^k^-k >~ implies 

max £ T A fc £ > for any £ ^ 0. 

0<fc<m 

Let /U = min{m, maxj rank(^4jX)}. We have the following estimate of the SDP approximation ratio. 
Theorem 4.2. There holds 

^ ax <^T< 21 °g( 174 ^)^T x - 

Proof. Notice that C = Q T U T CUQ is diagonal and hence 

x(0 T Cx(C) = \— z ?qT U t CU Q(: = L__Tr (CX). 



max £ Ai.£ max £ Ah£ 

0<k<m 0<k<m 



Therefore for any a > 1 we have 



Prob i x(0 T Cx(£) >-Tr (CX) 
a 



Prob <j max £ T A/%£ < a 

0<k<m 



1 — Prob < max £ .Aj.£ > a 

0<fc<m 



> 1 - Prob i max £ T i fc £ > a \ - Prob |£ T i £ > a) . 

[l<fc<m J I J 

Since Tr (Ao) < 1 and so a — Tr (.Ao) > 0, it follows from Lemma |4. II that 

Prob {£ T i £ > a} < Prob J ^ (4>)tf£i& > I < 1 - ^ 

[l<«<i<m J 

Since y for A; = 1, ...,m, and Tr (A^) < 1, it follows from (12) in [9j that 



T ~ ■ ' 



Hence we have 



Prob <j max £ A^S, > a }> < 2m/j,e '-' 

Kk<m 



Prob { x(£) T Cx(£) > - Tr (CX) \ > ^- - 2m/xe~K 



a v ' J 87 

Letting a = 21og(174m//) ensures the above probability to be positive. Therefore, there exists a 
random vector £ such that Tr (CX) < a x(£) T Cx(£), and the theorem is proven. □ 

We point out that Theorem l4.2l is an improvement of the so-called approximate 5-Lemma of Ben-Tal, 
Nemirovski, and Roos [2] (Lemma A. 6). In particular, Ben-Tal et al. showed that a < 21og(16n 2 m/j,), 
in contrast to our bound a < 2 log(174m//). 
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Notice that in (|4,ip there is only one indefinite inequality. A natural question arises: can we allow 
more? The following example shows that the answer is "no" if we wish to have a data-independent 
worst-case approximation ratio. (Data-dependent approximation ratio bounds will be discussed in 
Section [5] where we do allow multiple indefinite constraints.) 



Example 4.3. Consider 

2 12 
S.t. MX\X2 + x\ < 1 

— Mx\x 2 + x\ < 1 
M(x\-xl) < 1, 

where M > is an arbitrarily large positive constant. Its SDP relaxation is 
max 



aax Xn + jjX 2 2 

s.t. MX 12 + X 22 < 1, -MX 12 + X 22 < 1, M(Xn - X 22 ) < 1 
X n X 12 1 ^ Q 
X 2 i x 22 

For this quadratic program, the first two constraints imply that |xiX2| < 1 A ^ 2 < jj and so x\ < M l x v ■ 

The third inequality assures that x\ < jj+x^- Therefore, x\ < min { M , + x i} — ^ ra ^s^- 
Moreover, x\<l, and so u™ ax < 2^8. 

The SDP relaxation satisfies both primal and dual Slater conditions, so the primal-dual optimal 
solutions exist. A feasible solution for the SDP relaxation (primal problem) is the 2 by 2 identity 
matrix, with the objective value being 1 + -A? > 1. On the other hand, since X 22 < M\X\ 2 \ -\-X 22 < 1, 
and Xn < X 22 + jj, an upper bound for the SDP optimal value is 1 + Therefore, for this example, 



the ratio ^ > ^fjs - 0.382M, which can be arbitrarily large, depending on the size of M. 

If there are at most two homogeneous quadratic constraints, and moreover if the SDP relaxation 
has a primal-dual complementary optimal solution, then the SDP optimal value will be equal to the 
optimal value of the quadratic model; see e.g. Ye and Zhang [12] (Corollary 2.6). In other words, if 
there are no more than two inequality constraints, then under the primal-dual Slater condition, we 
will have v^j^/v^ x = 1. In this sense, Example 14.31 is the smallest possible in size. By removing the 
requirement that the SDP relaxation has a finite optimal value, then it is possible to construct an 
example which involves only two inequality constraints. 



Example 4.4. Consider 



max x±x 2 + x\ 
s.t. X\X 2 < 1 

X? - Xn < 1, 
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with the SDP relaxation 



max 
s.t. 



X 12 + Xxx 

X\2 < 1, Xn — X22 < 1, 

Xll X\2 

X21 X22 



h 0. 



In terms of polar coordinates, (xi,X2) 
turned into 

max 
s.t. 



(r cos 9, r sin 9), the original quadratic problem can be 



r 2 (sin26» + cos 29 + l)/2 



r 2 sin 29 < 2 
r 2 cos 29 < 1. 

By a further change of variables (r 2 cos 29, r 2 sin 20) 



(7/1,2/2)) we can reformulate the original 



quadratic problem as 



max 
s.t. 



yi + y2 + vyf+yf 
yi < 2 
y 2 < 1. 

This optimization problem has a unique optimal solution at (2/1,2/2) = (2 5 1) with the optimal value 
being 3+ 2 v ^ ~ 2.618. The SDP relaxation problem is clearly unbounded, as any positive multiple of 
the identity matrix is feasible. Therefore, v^^/v^ x = +00. This example is possible because the 
dual of the SDP relaxation problem is infeasible. 



5 Quadratic Optimization with Multiple Indefinite Constraints 

Unlike the minimization form (jl.ip for which the SDP approximation ratio can be infinite when there 
are more than one indefinite constraints (see Example 13. 7p . the maximization form (|1.2p can still 
admit a finite SDP approximation ratio in this case. In particular, consider a general homogeneous 
quadratic maximization problem 

max x T Cx 

s.t. x T AkX < 1, k = 0, 1, m (5.1) 
x E F n . 

Suppose that I, T> are two index sets, X U T> = {0, 1, m} and X n T> = 0, such that y for 
k £ T> and indefinite for k £ X. The SDP relaxation for (|5.ip is 



max Tr(CX) 

s.t. Tr (A k X) < 1, fc = 0, 1, m (5.2) 

x y 0. 

We begin our analysis with a technical lemma which bounds the probability of an exponential tail. 
Similar bounds exist in the literature, e.g. [3]. However, the lemma below serves our needs exactly; 
for completeness we include a proof here. 
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Lemma 5.1. Let {Aj}™ =1 be any given real numbers and {Tji}2=i be i.i.d. random variables drawn 
from either the real or complex valued zero mean Gaussian distribution with unit variance. Let 



a = \j'Y^i=i A? an d $ = niax{max{Aj | 1 < i < n}, 0}. Then, for any a > there holds 

exp (— min{a!, ^} ^) , if m ~ -/V(0, 1) zs real Gaussian, 
exp (—min {a, ^} , i/r?j ~ A^ c (0, 1) is complex Gaussian. 



Prob i ^ Air/, 2 - ^ Ai > acr I < 



^ i=i i=i 

Proof. We will only prove the real Gaussian case; the complex case is similar and therefore omitted. 
Let P := |min{^,^}. Then, 2/3Aj < 1/2 for all i = l,...,n, and j3a = | min{j, a}. Note that for 
any t < 1/2 the following inequality holds: 

T^< e i+t2 - (5-3) 
Let C := e^^-* i=1 Air? ?. Since {r/^}™ =1 are standard i.i.d. x 2 random variables, it follows that 

E (o - n E ( e ^) . n = (n < (ip— ?) = ^ a - 

where the inequality is due to f)5.3[) . This together with the Markov inequality implies 

n n 



Prob <j *iVi " Yl Al - aa \ = Prob - e /3(QCT+E -i Al) } 

E(C) 



, i=l i=l 



< 



e /3(a CT +E™ =1 A,) 
< e 2f3 2 a 2 -0aa = e (3a(2f3cr-a) < & Pa(%-a) 

= e - min { Q 'f}f. 

The lemma is proven. □ 



We are now ready to pursue the performance analysis for the real case F = M. Assume that (|5.2p 
has an optimal solution X. Denote the set of (real) eigenvalues of A^X as Af, A^, k = 0, 1, m. 
Since Tr(A k X) < 1, it follows that £? =1 \f < 1. Moreover, ||AjfcX[|J. = £™ =1 (Af) 2 ! . fc = 0,l,...,m, 
where || • \\f denotes the Frobenius norm of a matrix. 

Let £ be a random vector drawn from the Gaussian distribution iV(0, X). For any a > 1 and 
< /c < m, we consider the probability of the event Prob{£ T ^4fc£ > a}. By diagonalization, we have 
Prob {£ T ^4fc£ > a} = Prob {Ya=1 ^H r ii > a i> wnere r/ = (r/i, r/ n ) T is a random vector following the 
normal distribution N(0,I n ). 



If we let a k := yE"=i(\ fc ) 2 = PfcAC|| F , and 5 fc := max{0,max{Af | 1 < i < n}} , then Lemma O 
leads to 

/ ( n — V n \ k a k ^ n — V n A fc \ 
Prob{£ T ^>a}<exp(-min| ^pL^,_j gf 1 - 1 ) , V0<Km. (5.4) 
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Alternatively, we can bound the tail probability using Chebyshev's inequality. In particular, since 
Var (E"=i X iVi) = 2 J2'i=i( X i) 2 = 2 \\ A kX\\ 2 F , it follows from Chebyshev's inequality 

( n ~| ( n n n ^ 

prob E A ^ 2 >« = Prob E A ^ 2 -E A * >«-E A * 



. 1=1 



i=l 
n 



< Prob 



n 



E«-E A 



i=i 

Var(£r=iAk 2 ) 2 p fc X||| 



j=i 



i=i ; 

n 

> a - E X i 

1=1 



< 



< 



-, V < /c < m, 



(«-E?=iA?) 2 " («-i) 2 

where we have used the fact a > 1 > EiLi X i ■ Applying Lemma [3711 and using (|5.5p ~ (|5.4p gives 
Prob {t T A k £ <a,k = 0,l,...,m; £ T C£ > Tr((7i:)} 

m 

> 1 - Prob {t T C£ < Tr (CX)} - Prob {£ T ,4fc£ > a} 



(5.5) 



fc=0 



> 



100 



E 

fc=0 



mm < exp I — mm 



8a k 



2 

If 



(a - iy 



Notice that S k < a k and Ei=i Af < 1 for any k. Therefore, we have, for any a > 1, 

Prob [fA k £<a, k = 0,1, ...,m; £ T C£>Tr(CX)} 



^ ToTD-E ex P 



a — 1 ] a-1 
mm < ■, — , 1 



min < exp [ — mm 



a" 



8a k 



a-1 J a - 1\ 211^X11! 



(T 



8a k J ' (a - l) 2 



Let us choose 



a = 1 + max < 



20 + 81og|£>|,min< 



(20 + 81og | J|) max p fe X|| F , J 200 ^ || A fc X| 



Since a < EI=i A* < 1 for E D, it follows from the choice of a that 



a — 1 ] a-1 
exp ( -mm<; — ^— , 1 



a-l\ / a-l\ 1 
ex P I — o ir~ < exp ( - — I < , VkGP, 



8cj fe 



100IPI 



and 



min <|exp ^— min -| 



a-1 J a - 1 \ 2p fc X||| | I 



CJ 



8a k J' (a- l) 2 



< 



100 
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This further implies that 

Prob [i T A k i<a, fc = 0,l,...,m; £ T C£ > Tr (CX)} > 

Summarizing, we obtain the following worst-case performance ratio bounds on the SDP relaxation for 
a real-valued homogeneous (indefinite) quadratic maximization problem. [We also state the complex 
case without proof.] 

Theorem 5.2. For the quadratic optimization problem (\5.3 with F = R and its SDP relaxation 
(|5.#p . suppose that an optimal solution, say X, for (|5.^|) exists. Then, 



sdp 



qp 



< 1 + max { 20 + 8 log \V\ , min < 



(20 + 81og|J|)max||A fc X|| F , 200^2\\A k X\ 



Similarly, for the complex case F = C, we have 



sdp 



QP 



< 1 + max < 15 + 4 log \D\ , min < 



;i5 + 41og|T|)maxp fc X|| F , /40^ \\A k X\ 
fceX V hex 



Let us consider two special cases of Theorem 15.21 First, if T = 0, then Theorem 15.21 becomes 

^max 

-^1? < 20 + 81ogm (in the real case), which recovers the approximation result of Nemirovski et 
al. [9]. The second case is V = 0, where Theorem 15.21 becomes 



sdp 



QP 



< 1 + min { (20 + 81og(m + 1)) max [|A fc X||i?, 

0<fc<m 



200^p fe X| 



k=0 



Below is an example showing that this bound is also tight (in the order of magnitude). Specifically, 
consider Example 14.31 again: 

2 12 

max X-^ ~\~ ~^x<2 
S.t. MX\X2 +x\ < 1 

-MX\X2 + x\ < 1 

M{x\-xl) < 1. 

In this case we know that the SDP relaxation has an optimal solution X - 



1 + 







M 

1 



, while the 



approximation ratio is v^T^ /v™ x = O(M). There are three constraints, all indefinite, Z = {1,2,3}, 
with 



M 

u 2 
M -i 
2 1 



and so one may compute that 




A,X 



AL 

2 

K + I i 

2^2 x 



M 

2 





M _ 1 

" 2 2 



M 


,4j = 


M 





2 


1 







— M 



M 
2 



, A 3 X 



M + l 
—M 



Thus, ||A fc X||f, = 0(M 2 ), for fc = 1,2,3. Theorem predicts that vff* /v™ x < O(M), and this 
upper bound is exactly attained in this example. 
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6 Simulations and Discussions 



This paper studies the quality bound of SDP relaxation for solving nonconvex quadratic optimization 
problems (|3.ip and (|4.ip . For problem (|3.ip . a quality bound 0(m 2 ) was derived for F = R, and a 
quality bound 0{m) for F = C, when there is only one constraint x*Aqx > 1 with indefinite. For 
problem (|4.ip . a quality bound O(logm) was derived when there is only one nonconvex constraint 
x*Aqx < 1 with Aq indefinite. These quality bounds are independent of the problem dimension n or 
data matrices, and only depend on the number of constraints. 

For problem (13. ip . if there are two or more constraints in the form of x* Ax > 1 with A indefinite, 
then there is no general quality bound as shown by Example 13.71 For problem (|4.1|) , if there are 
two or more nonconvex constraints, a quality bound is given in Theorem 15.21 albeit the bound is 
dependent not only on the number of constraints but also the data of the problem. 

As shown in the preceding sections, these quality bounds are derived based on the worst-case analysis, 
and they are indeed tight, in the worst case, up to some constant. This analysis is important as a 
theoretical guide. The empirical tests, on the other side, serve a quite different purpose. Next, we 
present some numerical experiments on randomly generated instances. These numerical experiments 
show that the average approximation ratios are much better than the worst-case ratio, even though 
they appear to still follow the same growth trend (as a function of m). 

More specifically, we generate various random symmetric matrices A\. in the following way: for a full 
rank positive semidefinite Ak, we set Ak = rand-Q T -diag(abs(randn(n, 1))) -Q, where 'rand', 'randn' 
are Matlab notations, and Q is an orthogonal matrix obtained by QR factorization of a random matrix 
randn(re); for a rank-one positive semidefinite Ak, we set A}. = rand - Q T ■ diag([abs(randn); zeros(n — 
1, 1)]))-Q; and for an indefinite Ak, we set Ap. = rand-Q T -diag(randn(n, 1))-Q (Q defined as before). 
To examine the performance of SDP relaxation for randomly generated problem of form (|3.ip with 
F = R, we simply set C to be the identity matrix: fix n = 10, and choose m from 5, 10, 15, 100. 
For each m, we do the following: (a) Generate 1, 000 random problems such that only one of the A^s 
is indefinite, and all the other A^s are positive definite; (b) Generate 1,000 random problems such 
that 10% of the Ak's are indefinite, and all the other A^s are positive definite; (c) Generate 1,000 
random problems such that only one of the A^s is indefinite, and all the other Ak are rank one and 
positive semidefinite; (d) Generate 1,000 random problems such that 10% of the Ak's are indefinite, 
and all the other Ak are rank one and positive semidefinite. For each instance of the above randomly 
generated problems, we solve its SDP relaxation to obtain an optimal solution Z* and optimal value 
v sdp- Then we find one approximate solution for (|3.ip by the following randomization process. 
Generate 100 random vectors £ x , £ 100 . For each k = 1,...,100, let x % = £ l / \/ m.m k n =z ^ i ) T A^ 1 ■ 
Then w™ in < £™ in := min|£° {x l ) T Cx l . We use the empirical quality bound £™ in /vfj£ to estimate the 
real quality bound v^p^/v™^, since the former is at least the latter. These empirical quality bounds 
are plotted in Figure [TJ 
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(a): full rank with one indefinite 
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(c): one indefinite while others rank one 
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(b): full rank with 10% indefinite 
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(d): 10% indefinite while others rank one 
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Figure 1: Empirical quality bounds for problem p. II) with F = K 



In this figure, diamonds o are the maximum quality bounds in the 1,000 random problems for each 
m, stars * are the mean quality bounds, and circles o are the minimum quality bounds. For cases 
(a) and (c), we have quality bound 0(m 2 ) for the worst case, while for cases (b) and (d), there is no 
worst-case theoretical quality bound. As we can see, the computed empirical bounds are very small 
for case (a), and are moderate for case (c), and are indeed big for cases (b) and (d). 

To examine the SDP relaxation performance for the maximization problem (|4.ip . we generate four 
classes of random problems in the same way as for problem (13, ip . except that the matrix C in the 
objective is now indefinite (generated the same way as an indefinite A^). After the SDP relaxation 
is solved, we apply a similar randomization procedure to find a lower bound ?)™ ax for v™ ax as we did 
for problem (|3.ip . The empirical quality bound v™J^/v™ x is an upper bound for the actual quality 
bound ^pV^qp 1 *- The empirical quality bounds are plotted in Figure [21 The legends o, * and o carry 
the same meaning as they did in Figure [TJ This figure shows that the computed empirical bounds are 
close to one for cases (a) and (c), and are somewhat larger for cases (b) and (d). This is consistent 
with the bounds in Theorem 15.21 
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(a): full rank with one indefinite 



(b): full rank with 10 % indefinite 
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(c): one indefinite while others rank one 
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Figure 2: Empirical quality bounds for problem (|4,ll) 
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